{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "c5f7fce3",
      "metadata": {
        "id": "c5f7fce3"
      },
      "source": [
        "# 第三章 LangChain 表达式语言 LangChain Expression Language\n",
        "\n",
        "在这一章我们会介绍 LangChain Expression Language（或称为 LCEL），被称之为 Langchain 的表达式语言。LCEL 是一种新的语法，是 LangChain 工具包的重要补充，他有许多优点，使得我们处理 LangChain 和代理更加简单方便。\n",
        "\n",
        "1. LCEL 提供了异步、批处理和流处理支持，这使得代码多功能化，并且代码可以快速在不同服务器中应用和运行。\n",
        "    - 异步：程序可以同时执行多个任务，而不是按照顺序一个接一个地执行\n",
        "    - 批处理：是一种将一组任务或数据作为一个批次进行处理的方法，而不是逐个处理\n",
        "    - 流式处理：理是一种连续处理数据的方法，数据会持续不断地进入系统并被处理，流式处理能够在数据到达时立即进行处理，并且可以以持续且低延迟的方式处理数据。\n",
        "\n",
        "2. LCEL 拥有 fallbacks 措施，也叫回退安全机制，有时LLM得到的结果不可控，这时你可以将结果进行回退，甚至可以附加到整个链上\n",
        "\n",
        "3. LCEL 增加了 LLM 的并行性，LLM 运行通常是耗费时间的，并行可以加快得到结果的速度。\n",
        "\n",
        "4. LCEL 内置了日志记录，记录代理的运行情况。即使代理复杂，日志也有助于理解复杂链条和代理的运行情况。\n",
        "\n",
        "在前面的课程中，我们知道 LangChain 提供了组件 链（chain） 可以将组件组合起来发挥 LLM 更强大的功能，但是语法非常复杂。在这里，LCEL 提供了一种管道语法，使从基本组件构建复杂链变得容易，我们可以通过 LangChain 完成`Chain = prompt | LLM |OutputParser `的组合，具体使用我们将在下文内容中讨论。链（Chains）通常将大语言模型（LLM）与提示（Prompt）结合在一起，基于此，我们可以对文本或数据进行一系列操作。\n",
        "\n",
        "![image.png](../../figures/LCEL.png)\n",
        "\n",
        "- [一、简单链 Simple Chain](#一、简单链-Simple-Chain)\n",
        "- [二、更复杂的链 More complex chain](#二、更复杂的链-More-complex-chain)\n",
        "  - [1.1 构建简单向量数据库](#1.1-构建简单向量数据库)\n",
        "  - [1.2 使用RunnableMap](#1.2-使用RunnableMap)\n",
        "- [三、绑定 Bind](#三、绑定-Bind)\n",
        "  - [3.1 单函数绑定](#3.1-单函数绑定)\n",
        "  - [3.2 多个函数绑定](#3.2-多个函数绑定)\n",
        "- [四、后备措施 Fallbacks](#四、后备措施-Fallbacks)\n",
        "  - [4.1 使用早期模型格式化输出](#4.1-使用早期模型格式化输出)\n",
        "  - [4.2 使用新模型格式化输出](#4.2-使用新模型格式化输出)\n",
        "  - [4.3 fallbacks方法](#4.3-fallbacks方法)\n",
        "- [五、接口 Interface](#五、接口-Interface)\n",
        "  - [5.1 invoke接口](#5.1-invoke接口)\n",
        "  - [5.2 batch接口](#5.2-batch接口)\n",
        "  - [5.3 stream接口](#5.3-stream接口)\n",
        "  - [5.4 异步接口](#5.4-异步接口)\n",
        "- [六、英文版提示](#六、英文版提示)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b631e459",
      "metadata": {},
      "source": [
        "## 一、简单链 Simple Chain\n",
        "\n",
        "接下来我们依旧会使用 OpenAI 的 API，所以首先我们要初始化我们的 API_Key，这个方法和上一章的方式是一样的。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cacb572a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cacb572a",
        "outputId": "1acef39b-494e-445e-bde0-2e8f83a2a84c"
      },
      "outputs": [],
      "source": [
        "# !pip install langchain\n",
        "# !pip install openai==0.28\n",
        "# !pip install \"langchain[docarray]\"\n",
        "# !pip install tiktoken"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "id": "c52952e6",
      "metadata": {
        "id": "c52952e6"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import openai\n",
        "\n",
        "os.environ['OPENAI_API_KEY'] = \"YOUR_API_KEY\"\n",
        "openai.api_key = os.environ['OPENAI_API_KEY']"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0a7f48e9",
      "metadata": {
        "id": "0a7f48e9"
      },
      "source": [
        "接下来首先导入 LangChain 的库，并且定义一个简单的链，这个链包括提示模板，大语言模型和一个输出解析器。我们可以看到，成功输出了大语言模型的结果，完成了一个简单的链。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "id": "81ac70fe",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 91
        },
        "id": "81ac70fe",
        "outputId": "00302a8a-42eb-4eaa-ff0f-8540245932cd"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class `langchain_community.chat_models.openai.ChatOpenAI` was deprecated in langchain-community 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run `pip install -U langchain-openai` and import as `from langchain_openai import ChatOpenAI`.\n",
            "  warn_deprecated(\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'为什么熊不喜欢玩扑克牌？因为他总是把两个熊掌都露出来！'"
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 导入LangChain所需的模块\n",
        "from langchain.prompts import ChatPromptTemplate\n",
        "from langchain.chat_models import ChatOpenAI\n",
        "from langchain.schema.output_parser import StrOutputParser\n",
        "\n",
        "# 使用 ChatPromptTemplate 从模板创建一个提示，模板中的 {topic} 将在后续代码中替换为实际的话题\n",
        "prompt = ChatPromptTemplate.from_template(\n",
        "    \"告诉我一个关于{topic}的短笑话\"\n",
        ")\n",
        "\n",
        "# 创建一个 ChatOpenAI 模型实例，默认使用 gpt-3.5-turbo 模型\n",
        "model = ChatOpenAI()\n",
        "\n",
        "# 创建一个StrOutputParser实例，用于解析输出\n",
        "output_parser = StrOutputParser()\n",
        "\n",
        "# 创建一个链式调用，将 prompt、model 和output_parser 连接在一起\n",
        "chain = prompt | model | output_parser\n",
        "\n",
        "# 调用链式调用，并传入参数\n",
        "chain.invoke({\"topic\": \"熊\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7563f727",
      "metadata": {
        "id": "7563f727"
      },
      "source": [
        "如果我们去查看`Chain`的输出，我们会发现，他跟我们定义的是一样的，一共有三部分进行组成，也就是`Chain = prompt | LLM |OutputParser `。`|`符号类似于 unix 管道操作符，它将不同的组件链接在一起，将一个组件的输出作为输入提供给下一个组件。在这个链中，用户输入被传递给提示模板，然后提示模板输出被传递给模型，然后模型输出被传递到输出解析器。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "a351d14a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a351d14a",
        "outputId": "e9a7f195-fdfb-4523-eb34-a708ce0e0b24"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "ChatPromptTemplate(input_variables=['topic'], messages=[HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['topic'], template='告诉我一个关于{topic}的短笑话'))])\n",
              "| ChatOpenAI(client=<class 'openai.api_resources.chat_completion.ChatCompletion'>, openai_api_key='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', openai_proxy='')\n",
              "| StrOutputParser()"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 查看Chain的值\n",
        "chain"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "35f8c80e",
      "metadata": {
        "id": "35f8c80e"
      },
      "source": [
        "## 二、更复杂的链 More complex chain\n",
        "\n",
        "接下来，我们会创建一个更复杂的链条，在之前的课程中，我们接触过如何进行检索增强生成。所以接下来我们使用 LCEL 来重复之前的过程，将用户的问题和向量数据库检索结果结合起来，使用 RunnableMap 来构建一个更复杂的链。\n",
        "\n",
        "### 2.1 构建简单向量数据库\n",
        "首先我们构建一个向量数据库，这个简单的向量数据库只包含两句话，使用 OpenAI 的 Embedding 作为嵌入模型，然后我们通过 `vector store.as_retriever `来创建一个检索器。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "id": "308eab74",
      "metadata": {
        "id": "308eab74"
      },
      "outputs": [],
      "source": [
        "from langchain.embeddings import OpenAIEmbeddings\n",
        "from langchain.vectorstores import DocArrayInMemorySearch\n",
        "\n",
        "# 创建一个DocArrayInMemorySearch对象，用于存储和搜索文档向量\n",
        "vectorstore = DocArrayInMemorySearch.from_texts(\n",
        "    [\"哈里森在肯肖工作\", \"熊喜欢吃蜂蜜\"],\n",
        "    embedding=OpenAIEmbeddings() # 使用OpenAI的Embedding\n",
        ")\n",
        "\n",
        "# 创建一个检索器\n",
        "retriever = vectorstore.as_retriever()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "768c1f68",
      "metadata": {
        "id": "768c1f68"
      },
      "source": [
        "通过之前的学习，如果我们调用`retriever.get_relevant_documents`，我们会得到相关的检索文档，首先我们问“哈里森在哪里工作？”，我们会发现返回了一个文档列表，他会根据相似度排序返回文档列表，所以其中最相关的放在了第一个。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "id": "f83e9118",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f83e9118",
        "outputId": "4b3e4b1f-c378-43ec-8a2e-4b6a6a0d88b0"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[Document(page_content='哈里森在肯肖工作'), Document(page_content='熊喜欢吃蜂蜜')]"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 获取与问题“哈里森在哪里工作？”相关的文档\n",
        "retriever.get_relevant_documents(\"哈里森在哪里工作？\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a4b5eacb",
      "metadata": {
        "id": "a4b5eacb"
      },
      "source": [
        "如果我们换一个问题，比如\"熊喜欢吃什么\"，可以看到问题的顺序就发生了变化。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "id": "7d76aaf5",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7d76aaf5",
        "outputId": "8e79e2d9-8572-44e0-edf7-70cd254e0252"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[Document(page_content='熊喜欢吃蜂蜜'), Document(page_content='哈里森在肯肖工作')]"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 获取与问题“熊喜欢吃什么”相关的文档\n",
        "retriever.get_relevant_documents(\"熊喜欢吃什么\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fb34293a",
      "metadata": {
        "id": "fb34293a"
      },
      "source": [
        "### 3.2 使用RunnableMap\n",
        "\n",
        "上述例子返回两个结果是因为只有两个文档列表，这完全适用于更多文档情况。接下来我们会加入`RunnableMap`，在这个`RunnableMap`中，不仅仅有用户的问题，以及有对应的问题的文档列表，相当于这也为大模型的文档增加了上下文，这样就能完成检索增强的事情。如果我们正常问一个问题，可以看到，大模型正确的返回了文档里面的结果，得到了正确的输出。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "id": "e8a9b652",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "e8a9b652",
        "outputId": "d615b86a-a14b-4917-833f-e70c91730477"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'肯肖'"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from langchain.schema.runnable import RunnableMap\n",
        "\n",
        "# 定义一个模板字符串template\n",
        "template = \"\"\"仅根据以下上下文回答问题：\n",
        "{context}\n",
        "\n",
        "问题：{question}\n",
        "\"\"\"\n",
        "\n",
        "# 使用 template 作为模板\n",
        "prompt = ChatPromptTemplate.from_template(template)\n",
        "\n",
        "# 创建一个处理链 chain ，包含了 RunnableMap、prompt、model 和 output_parser 组件\n",
        "chain = RunnableMap({\n",
        "    \"context\": lambda x: retriever.get_relevant_documents(x[\"question\"]),\n",
        "    \"question\": lambda x: x[\"question\"]\n",
        "}) | prompt | model | output_parser\n",
        "\n",
        "# 调用chain的invoke方法\n",
        "chain.invoke({\"question\": \"哈里森在哪里工作?\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "394af88e",
      "metadata": {
        "id": "394af88e"
      },
      "source": [
        "如果我们想更深入挖掘一下背后的工作机理，我们可以看一下`RunnableMap`，我们把其创建为一个输入，用一样的方式进行操作。我们可以看到，在这之中，`RunnableMap`提供了`context`和`question`两个变量，一个是查询的文档列表，另一个是对应的问题，这个大模型就可以根据提供文档来总结回答对应的问题了。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "id": "759d2af6",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "759d2af6",
        "outputId": "fa16fa6c-dcb0-46e9-e538-428e711db372"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'context': [Document(page_content='哈里森在肯肖工作'),\n",
              "  Document(page_content='熊喜欢吃蜂蜜')],\n",
              " 'question': '哈里森在哪里工作?'}"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 创建一个RunnableMap对象，其中包含两个键值对\n",
        "# 键 \"context\" 对应一个lambda函数，用于获取相关文档，函数输入参数为x，即输入的字典，函数返回值为retriever.get_relevant_documents(x[\"question\"])\n",
        "# 键 \"question\" 对应一个lambda函数，用于获取问题，函数输入参数为x，即输入的字典，函数返回值为x[\"question\"]\n",
        "inputs = RunnableMap({\n",
        "    \"context\": lambda x: retriever.get_relevant_documents(x[\"question\"]),\n",
        "    \"question\": lambda x: x[\"question\"]\n",
        "})\n",
        "\n",
        "# 调用 inputs 的 invoke 方法，并传递一个字典作为参数，字典中包含一个键值对，键为\"question\"，值为\"哈里森在哪里工作?\"\n",
        "inputs.invoke({\"question\": \"哈里森在哪里工作?\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2c01b704",
      "metadata": {
        "id": "2c01b704"
      },
      "source": [
        "## 三、绑定 Bind\n",
        "\n",
        "在上一章我们介绍了OpenAI函数的调用，新的`function`参数可以自动判断是否要使用工具函数，如果需要就会返回需要使用的参数。接下来我们也使用LangChain实现OpenAI函数调用的新功能，首先需要一个函数的描述信息，以及定义函数，这里的函数还是使用上一章的`get_current_weather`函数。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "id": "1b00f651",
      "metadata": {
        "id": "1b00f651"
      },
      "outputs": [],
      "source": [
        "# 定义一个函数\n",
        "functions = [\n",
        "  {\n",
        "    \"name\": \"get_current_weather\",\n",
        "    \"description\": \"获取指定位置的当前天气情况\",\n",
        "    \"parameters\": {\n",
        "      \"type\": \"object\",\n",
        "      \"properties\": {\n",
        "        \"location\": {\n",
        "          \"type\": \"string\",\n",
        "          \"description\": \"城市和省份，例如：北京，北京市\",\n",
        "        },\n",
        "        \"unit\": {\"type\": \"string\", \"enum\": [\"摄氏度\", \"华氏度\"]},\n",
        "      },\n",
        "      \"required\": [\"location\"],\n",
        "    },\n",
        "  }\n",
        "]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5d2ff71d",
      "metadata": {
        "id": "5d2ff71d"
      },
      "source": [
        "### 3.1 单函数绑定\n",
        "\n",
        "接下来我们使用`bind`的方法把工具函数绑定到大模型上，并构建一个简单的链。进行调用以后，我们可以看到返回了一个`AIMessage`，\t其中返回的`content`为空，但是返回了我们需要调用工具函数的参数。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "id": "2f693ff9",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2f693ff9",
        "outputId": "9ebdd754-906f-4178-a31a-899ed3c7d304"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_current_weather', 'arguments': '{\\n  \"location\": \"北京，北京市\"\\n}'}})"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 使用ChatPromptTemplate.from_messages方法创建一个ChatPromptTemplate对象\n",
        "prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"human\", \"{input}\")\n",
        "    ]\n",
        ")\n",
        "\n",
        "# 使用bind方法绑定functions参数\n",
        "model = ChatOpenAI(temperature=0).bind(functions=functions)\n",
        "\n",
        "runnable = prompt | model\n",
        "\n",
        "# 调用invoke方法\n",
        "runnable.invoke({\"input\": \"北京天气怎么样？\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "89a5fb09",
      "metadata": {
        "id": "89a5fb09"
      },
      "source": [
        "### 3.2 多个函数绑定\n",
        "\n",
        "同时我们也可以定义多个`function`，大模型在对话的时候可以自动判断使用哪一个函数。这里面我们定义有两个函数，第一个函数是类似于前面的`weather_search`，搜索给定机场的天气，然后我们还定义了一个赛事体育新闻搜索的`sports_search`，查询天气的函数`weather_search`接受的参数为airport_code即机场代码，体育新闻搜索函数`sports_search`接受的参数为team_name即体育队名。由于这里我们不需要运行这些函数，因为大模型是通过问的问题来自动判断是否调用这些函数，并且返回参数，并不会直接帮我们调用。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "id": "b95f4029",
      "metadata": {
        "id": "b95f4029"
      },
      "outputs": [],
      "source": [
        "functions = [\n",
        "    {\n",
        "        \"name\": \"weather_search\",\n",
        "        \"description\": \"搜索给定机场代码的天气\",\n",
        "        \"parameters\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": {\n",
        "                \"airport_code\": {\n",
        "                    \"type\": \"string\",\n",
        "                    \"description\": \"要获取天气的机场代码\"\n",
        "                },\n",
        "            },\n",
        "            \"required\": [\"airport_code\"]\n",
        "        }\n",
        "    },\n",
        "    {\n",
        "        \"name\": \"sports_search\",\n",
        "        \"description\": \"搜索最近体育赛事的新闻\",\n",
        "        \"parameters\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": {\n",
        "                \"team_name\": {\n",
        "                    \"type\": \"string\",\n",
        "                    \"description\": \"要搜索的体育队名\"\n",
        "                },\n",
        "            },\n",
        "            \"required\": [\"team_name\"]\n",
        "        }\n",
        "    }\n",
        "]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d4a77d53",
      "metadata": {
        "id": "d4a77d53"
      },
      "source": [
        "接着我们就可以使用函数绑定大模型，定义一个简单的链，我们可以看到，当我们问了相关的问题以后，大模型能够自动判断并且正确返回参数，知道需要调用函数了。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "id": "519e761d",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "519e761d",
        "outputId": "7ad43d65-bd62-4339-c8f8-e5c74a414dac"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "AIMessage(content='', additional_kwargs={'function_call': {'name': 'sports_search', 'arguments': '{\\n  \"team_name\": \"爱国者队\"\\n}'}})"
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 绑定大模型\n",
        "model = model.bind(functions=functions)\n",
        "runnable = prompt | model\n",
        "\n",
        "runnable.invoke({\"input\": \"爱国者队昨天表现的怎么样?\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0b988820",
      "metadata": {
        "id": "0b988820"
      },
      "source": [
        "## 四、后备措施 Fallbacks\n",
        "\n",
        "在使用早期的OpenAI模型如\"text-davinci-001\"，这些模型在对话过程中，不支持格式化输出结果即它们都是以字符串的形式输出结果，这对我们有时候需要解析 LLM 的输出带来一些麻烦，比如下面这个例子，就是利用早期模型\"text-davinci-001\"来回答用户的问题，我们希望 llm 能以 json 格式输出结果。\n",
        "\n",
        "我们定义了 OpenAI 的模型以及创建了一个简单的链，以此加入 json 希望能以 json 格式输出结果，我们让 simple_model 写三首诗，并以 josn 格式输出，每首诗必须包含:`标题，作者和诗的第一句`。我们会发现结果只有字符串，无法输出指定格式的内容，虽然里面有一些`[`，但是本质上还是一个大的字符串，这就无法让我们解析输出。\n",
        "\n",
        "> 由于OpenAI于2024年1月4日停用了模型text-davinci-001，你将使用OpenAI推荐的替代模型gpt-3.5-turbo-instruct。\n",
        "\n",
        "在使用语言模型时，你可能经常会遇到来自底层 API 的问题，无论这些问题是速率限制还是停机时间。因此，当你将 LLM 应用程序转移到实际生产环境中时，防范这些问题变得越来越重要。这就是为什么我们引入了`回退（Fallbacks）`的概念。"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "127d93a1",
      "metadata": {},
      "source": [
        "### 4.1 使用早期模型格式化输出"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "id": "11e2b2e2",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 180
        },
        "id": "11e2b2e2",
        "outputId": "1294676f-7fbf-4185-ac16-49fa5d0512f5"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/langchain_core/_api/deprecation.py:117: LangChainDeprecationWarning: The class `langchain_community.llms.openai.OpenAI` was deprecated in langchain-community 0.0.10 and will be removed in 0.2.0. An updated version of the class exists in the langchain-openai package and should be used instead. To use it run `pip install -U langchain-openai` and import as `from langchain_openai import OpenAI`.\n",
            "  warn_deprecated(\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'\\n\\n{\\n  \"title\": \"春风\",\\n  \"author\": \"李白\",\\n  \"first_line\": \"春风又绿江南岸\",\\n  \"content\": [\\n    \"春风又绿江南岸\",\\n    \"花开满树柳如丝\",\\n    \"鸟儿欢唱天地宽\",\\n    \"人间春色最宜人\"\\n  ]\\n}\\n\\n{\\n  \"title\": \"夜雨\",\\n  \"author\": \"杜甫\",\\n  \"first_line\": \"夜雨潇潇\",\\n  \"content\": [\\n    \"夜雨潇潇\",\\n    \"孤灯照旧\",\\n    \"思念如潮\",\\n    \"泛滥心头\"\\n  ]\\n}\\n\\n{\\n  \"title\": \"山行\",\\n  \"author\": \"王维\",\\n  \"first_line\": \"远上寒山石径斜\",\\n  \"content\": [\\n    \"远上寒山石径斜\",\\n    \"白云生处有人家\",\\n    \"停车坐爱枫林晚\",\\n    \"霜叶红于二月花\"\\n  ]\\n}'"
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from langchain.llms import OpenAI\n",
        "import json\n",
        "\n",
        "# 使用早期的OpenAI模型\n",
        "simple_model = OpenAI(\n",
        "    temperature=0,\n",
        "    max_tokens=1000,\n",
        "    model=\"gpt-3.5-turbo-instruct\"\n",
        ")\n",
        "simple_chain = simple_model | json.loads\n",
        "\n",
        "challenge = \"写三首诗，并以josn格式输出，每首诗必须包含:标题，作者和诗的第一句。\"\n",
        "\n",
        "simple_model.invoke(challenge)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6a22ae8e",
      "metadata": {
        "id": "6a22ae8e"
      },
      "source": [
        "如果我们使用`simple_chain`来运行，我们就会发现出现了 json 解码错误的问题，因为返回的结果就是一个字符串，无法解析，所以下面代码就会报错。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "id": "08ee6ba5",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "id": "08ee6ba5",
        "outputId": "2ef7f71f-4398-47ed-a8d8-bcaa75bd91d8"
      },
      "outputs": [
        {
          "ename": "JSONDecodeError",
          "evalue": "Extra data: line 15 column 1 (char 147)",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mJSONDecodeError\u001b[0m                           Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-17-7b2363c45b31>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msimple_chain\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minvoke\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchallenge\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/base.py\u001b[0m in \u001b[0;36minvoke\u001b[0;34m(self, input, config)\u001b[0m\n\u001b[1;32m   2051\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2052\u001b[0m             \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstep\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msteps\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2053\u001b[0;31m                 input = step.invoke(\n\u001b[0m\u001b[1;32m   2054\u001b[0m                     \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2055\u001b[0m                     \u001b[0;31m# mark each step as a child run\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/base.py\u001b[0m in \u001b[0;36minvoke\u001b[0;34m(self, input, config, **kwargs)\u001b[0m\n\u001b[1;32m   3505\u001b[0m         \u001b[0;34m\"\"\"Invoke this runnable synchronously.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3506\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"func\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3507\u001b[0;31m             return self._call_with_config(\n\u001b[0m\u001b[1;32m   3508\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_invoke\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3509\u001b[0m                 \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/base.py\u001b[0m in \u001b[0;36m_call_with_config\u001b[0;34m(self, func, input, config, run_type, **kwargs)\u001b[0m\n\u001b[1;32m   1244\u001b[0m             output = cast(\n\u001b[1;32m   1245\u001b[0m                 \u001b[0mOutput\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1246\u001b[0;31m                 context.run(\n\u001b[0m\u001b[1;32m   1247\u001b[0m                     \u001b[0mcall_func_with_variable_args\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1248\u001b[0m                     \u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m  \u001b[0;31m# type: ignore[arg-type]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/config.py\u001b[0m in \u001b[0;36mcall_func_with_variable_args\u001b[0;34m(func, input, config, run_manager, **kwargs)\u001b[0m\n\u001b[1;32m    324\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mrun_manager\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0maccepts_run_manager\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    325\u001b[0m         \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"run_manager\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrun_manager\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 326\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# type: ignore[call-arg]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    327\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    328\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/base.py\u001b[0m in \u001b[0;36m_invoke\u001b[0;34m(self, input, run_manager, config, **kwargs)\u001b[0m\n\u001b[1;32m   3381\u001b[0m                         \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mchunk\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3382\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3383\u001b[0;31m             output = call_func_with_variable_args(\n\u001b[0m\u001b[1;32m   3384\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconfig\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrun_manager\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3385\u001b[0m             )\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/config.py\u001b[0m in \u001b[0;36mcall_func_with_variable_args\u001b[0;34m(func, input, config, run_manager, **kwargs)\u001b[0m\n\u001b[1;32m    324\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mrun_manager\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0maccepts_run_manager\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    325\u001b[0m         \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"run_manager\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrun_manager\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 326\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# type: ignore[call-arg]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    327\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    328\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/lib/python3.10/json/__init__.py\u001b[0m in \u001b[0;36mloads\u001b[0;34m(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)\u001b[0m\n\u001b[1;32m    344\u001b[0m             \u001b[0mparse_int\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mparse_float\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    345\u001b[0m             parse_constant is None and object_pairs_hook is None and not kw):\n\u001b[0;32m--> 346\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0m_default_decoder\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    347\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mcls\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    348\u001b[0m         \u001b[0mcls\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mJSONDecoder\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/lib/python3.10/json/decoder.py\u001b[0m in \u001b[0;36mdecode\u001b[0;34m(self, s, _w)\u001b[0m\n\u001b[1;32m    338\u001b[0m         \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_w\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    339\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 340\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mJSONDecodeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Extra data\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    341\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mobj\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    342\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mJSONDecodeError\u001b[0m: Extra data: line 15 column 1 (char 147)"
          ]
        }
      ],
      "source": [
        "simple_chain.invoke(challenge)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1c534366",
      "metadata": {
        "id": "1c534366"
      },
      "source": [
        "### 4.2 使用新模型格式化输出\n",
        "\n",
        "所以我们会发现早期版本的 OpenAI 模型不支持格式化的输出，所以即使使用 LangChain 并且加上了`json.load`但是还是会出现错误，但是如果我们使用新的`gpt-3.5-turbo`模型就不会出现这个问题。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "id": "f0e34ed7",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f0e34ed7",
        "outputId": "f3ae66a2-b0c7-4557-8914-687ac20b2a08"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'poem1': {'title': '春风',\n",
              "  'author': '李白',\n",
              "  'first_line': '春风又绿江南岸。',\n",
              "  'content': '春风又绿江南岸，明月何时照我还。'},\n",
              " 'poem2': {'title': '静夜思',\n",
              "  'author': '杜甫',\n",
              "  'first_line': '床前明月光，',\n",
              "  'content': '床前明月光，疑是地上霜。'},\n",
              " 'poem3': {'title': '登鹳雀楼',\n",
              "  'author': '王之涣',\n",
              "  'first_line': '白日依山尽，',\n",
              "  'content': '白日依山尽，黄河入海流。'}}"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 默认使用新的模型\n",
        "model = ChatOpenAI(temperature=0)\n",
        "chain = model | StrOutputParser() | json.loads\n",
        "\n",
        "chain.invoke(challenge)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "603a2605",
      "metadata": {
        "id": "603a2605"
      },
      "source": [
        "### 4.3 fallbacks方法\n",
        "\n",
        "那这个时候可能就会思考，有没有什么方法，在不用改变太多代码的情况下，让早期的模型也能达到格式化输出的效果，而不是写复杂的格式化输出的代码去对结果进行操作。这时候我们就可以使用`fallbacks`的方式赋予早期模型这样格式化的能力，从结果我们也可以看出，我们成功使用`fallbacks`赋予了简单模型格式化的能力。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "id": "7b1aede1",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7b1aede1",
        "outputId": "3823946c-a875-4807-9b0d-ba2910fe584d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'poem1': {'title': '春风',\n",
              "  'author': '李白',\n",
              "  'first_line': '春风又绿江南岸。',\n",
              "  'content': '春风又绿江南岸，明月何时照我还。'},\n",
              " 'poem2': {'title': '静夜思',\n",
              "  'author': '杜甫',\n",
              "  'first_line': '床前明月光，',\n",
              "  'content': '床前明月光，疑是地上霜。'},\n",
              " 'poem3': {'title': '登鹳雀楼',\n",
              "  'author': '王之涣',\n",
              "  'first_line': '白日依山尽，',\n",
              "  'content': '白日依山尽，黄河入海流。'}}"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 使用with_fallbacks机制\n",
        "final_chain = simple_chain.with_fallbacks([chain])\n",
        "\n",
        "# 调用final_chain的invoke方法，并传递challenge参数\n",
        "final_chain.invoke(challenge)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bb6f0aac",
      "metadata": {},
      "source": [
        "### 4.4 fallbacks 是如何实现的？\n",
        "\n",
        "当我们调用 LLM 时，经常会出现由于底层 API 问题、速率问题或者网络问题等原因，导致不能成功运行 LLM 。在这种情况下，我们就可以使用回退这种方法来解决这个问题，具体来说，他是通过使用另一种 LLM 来代替原先的不可运行的 LLM 产生结果，请看下面例子："
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4dd3f251",
      "metadata": {},
      "outputs": [],
      "source": [
        "from langchain_core.chat_models.openai import ChatOpenAI\n",
        "from langchain_core.chat_models.anthropic import ChatAnthropic\n",
        "\n",
        "model = ChatAnthropic().with_fallbacks([ChatOpenAI()])\n",
        "model.invoke('hello')"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1b8fb2aa",
      "metadata": {},
      "source": [
        "在这种情况下，通常会优先使用 ChatAnthropic 进行回答，但是如果调用 ChatAnthropic 失败了，会回退到使用 ChatOpenAI 模型来生成响应。如果两种 LLM 都失败了，将会回退到一种硬编码响应。硬编码的默认响应用于处理异常情况或者在无法从外部资源获取所需信息时提供一个备用选项，例如 \"Looks like our LLM providers are down. Here's a nice 🦜️ emoji for you instead.\"（看起来我们的 LLM 提供商出了问题，那么，这里有一个可爱的 🦜️ 表情符号给你。）"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "438e1841",
      "metadata": {},
      "source": [
        "如果你想了解更多关于 fallbacks 的内容，请参考[官方文档](https://python.langchain.com/docs/guides/fallbacks)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "96e3229d",
      "metadata": {
        "id": "96e3229d"
      },
      "source": [
        "## 五、接口 Interface\n",
        "\n",
        "在使用LangChain中，存在许多接口，其中公开的标准接口包括：\n",
        "\n",
        "- stream：流式返回输出内容\n",
        "- invoke：输入调用chain\n",
        "- batch：在输入列表中并行调用chain\n",
        "\n",
        "这些也有相应的异步方法：\n",
        "\n",
        "- astream：异步流式返回输出内容\n",
        "- ainvoke：在输入上异步调用chain\n",
        "- abatch：在输入列表中并行异步调用chain\n",
        "\n",
        "首先我们定义给一个简单提示模板，也就是\"给我讲一个关于{主题}的短笑话\"，然后定义了一个简单的链`Chain = prompt | LLM | OutputParser`。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "id": "72e228da",
      "metadata": {
        "id": "72e228da"
      },
      "outputs": [],
      "source": [
        "# 创建一个ChatPromptTemplate对象，使用模板\"给我讲一个关于{topic}的短笑话\"\n",
        "prompt = ChatPromptTemplate.from_template(\n",
        "    \"给我讲一个关于{topic}的短笑话\"\n",
        ")\n",
        "\n",
        "# 创建一个ChatOpenAI模型\n",
        "model = ChatOpenAI()\n",
        "\n",
        "# 创建一个StrOutputParser对象\n",
        "output_parser = StrOutputParser()\n",
        "\n",
        "# 创建一个chain，将prompt、model和output_parser连接起来\n",
        "chain = prompt | model | output_parser"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e8b02924",
      "metadata": {
        "id": "e8b02924"
      },
      "source": [
        "### 5.1 invoke接口\n",
        "\n",
        "接下来我们分别使用对应的接口，比如我们首先使用常规的`invoke`的调用，这个也是前面展现的方法，我们得到了对应结果。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "id": "e339d019",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "e339d019",
        "outputId": "6893312f-8473-411f-d175-c351532e5646"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'当熊在森林里遇到一只兔子时，他问：“兔子先生，你有没有问题？”兔子回答道：“当然，先生熊，我有一个问题。你怎么会拉这么长的尾巴？”熊听后笑了起来：“兔子先生，这不是尾巴，这是我的领带！”'"
            ]
          },
          "execution_count": 21,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chain.invoke({\"topic\": \"熊\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2520f449",
      "metadata": {
        "id": "2520f449"
      },
      "source": [
        "### 5.2 batch接口\n",
        "\n",
        "我们再尝试使用`batch`的接口，我们会发现大模型可以返回两个问题的答案，我们会给chain一个输入的列表，列表中可以包含多个问题，最后返回多个问题的答案。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "id": "d549ac8a",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d549ac8a",
        "outputId": "fbd34c84-26b2-4576-d7ce-aa73be98d755"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['好的，这是一个关于熊的短笑话：\\n\\n有一天，一只熊走进了一家酒吧。他走到吧台前，对酒保说：“请给我一杯……蜂蜜啤酒。”\\n\\n酒保疑惑地看着熊，说：“对不起，我们这里没有蜂蜜啤酒。”\\n\\n熊有些失望地叹了口气，然后说：“好吧，那就给我来一杯……草莓酒吧。”\\n\\n酒保摇摇头，说：“抱歉，我们也没有草莓酒。”\\n\\n熊又叹了口气，然后说：“那请给我来一杯……蜜糖红酒吧。”\\n\\n酒保实在无法忍受了，他对熊说：“对不起，我们这里没有这些奇怪的酒，你是熊，你应该知道熊只能喝蜂蜜。”\\n\\n熊听后一愣，然后脸色一变，说：“原来你们这里没有蜂蜜啤酒，草莓酒和蜜糖红酒？那请给我来一杯……白开水吧。”',\n",
              " '有一天，一只狐狸在森林里遇到了一只兔子。狐狸笑嘻嘻地对兔子说：“喂，兔子，我有一个好消息和一个坏消息，你想先听哪个？”兔子有些好奇地问：“那就先告诉我好消息吧。”狐狸眯起眼睛说：“好消息是，你的智商比我高。”兔子高兴地跳了起来：“太好了，那坏消息是什么？”狐狸一脸轻松地说：“坏消息是，你的智商还比不过胡萝卜。”']"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chain.batch([{\"topic\": \"熊\"}, {\"topic\": \"狐狸\"}])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "307a41d2",
      "metadata": {
        "id": "307a41d2"
      },
      "source": [
        "### 5.3 stream接口\n",
        "\n",
        "接下来我们在看看`stream`接口，也就是流式输出内容，这样的功能很有必要，有时候可以免去用户等待的烦恼，让用户看到一个一个词蹦出来而不是一个空的屏幕，这样会带来更好的用户体验。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "id": "f934e46d",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f934e46d",
        "outputId": "872109e2-47cb-4da6-a439-eb010ed3530d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "好\n",
            "的\n",
            "，\n",
            "这\n",
            "是\n",
            "一个\n",
            "关\n",
            "于\n",
            "熊\n",
            "的\n",
            "短\n",
            "笑\n",
            "话\n",
            "：\n",
            "\n",
            "\n",
            "有\n",
            "一\n",
            "天\n",
            "，\n",
            "一\n",
            "只\n",
            "小\n",
            "熊\n",
            "走\n",
            "进\n",
            "了\n",
            "一\n",
            "家\n",
            "酒\n",
            "吧\n",
            "。\n",
            "他\n",
            "走\n",
            "到\n",
            "吧\n",
            "台\n",
            "前\n",
            "，\n",
            "对\n",
            "酒\n",
            "保\n",
            "说\n",
            "：“\n",
            "酒\n",
            "保\n",
            "，\n",
            "给\n",
            "我\n",
            "一\n",
            "杯\n",
            "牛\n",
            "奶\n",
            "。”\n",
            "\n",
            "\n",
            "酒\n",
            "保\n",
            "惊\n",
            "讶\n",
            "地\n",
            "问\n",
            "道\n",
            "：“\n",
            "小\n",
            "熊\n",
            "，\n",
            "你\n",
            "怎\n",
            "么\n",
            "会\n",
            "来\n",
            "这\n",
            "里\n",
            "喝\n",
            "牛\n",
            "奶\n",
            "？\n",
            "”\n",
            "\n",
            "\n",
            "小\n",
            "熊\n",
            "深\n",
            "情\n",
            "地\n",
            "回\n",
            "答\n",
            "：“\n",
            "因\n",
            "为\n",
            "我的\n",
            "妈\n",
            "妈\n",
            "说\n",
            "，\n",
            "每\n",
            "当\n",
            "我\n",
            "喝\n",
            "酒\n",
            "的\n",
            "时\n",
            "候\n",
            "，\n",
            "我\n",
            "都\n",
            "会\n",
            "变\n",
            "得\n",
            "熊\n",
            "样\n",
            "！”\n",
            "\n"
          ]
        }
      ],
      "source": [
        "for t in chain.stream({\"topic\": \"熊\"}):\n",
        "    print(t)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fda98771",
      "metadata": {
        "id": "fda98771"
      },
      "source": [
        "### 5.4 异步接口\n",
        "\n",
        "我们还可以尝试异步来调用，使用`ainvoke`来调用。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "id": "cd372fec",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "id": "cd372fec",
        "outputId": "1232ac0b-7900-493c-aec6-7b99abb928d8"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'好的，以下是一个关于熊的短笑话：\\n\\n有一只熊走进了一家餐厅，他走到柜台前，对着服务员说：“我想要一杯咖啡和......嗯，一块...牛肉三明治。”\\n服务员疑惑地看着熊，然后问道：“对不起，先生，你是真的想要一块牛肉三明治吗？”\\n熊点了点头。\\n服务员又问：“那请问为什么你要来这里点餐呢？”\\n熊回答道：“因为我是个熊啊！”'"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response = await chain.ainvoke({\"topic\": \"熊\"})\n",
        "response"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "926c23d0",
      "metadata": {
        "id": "926c23d0"
      },
      "source": [
        "## 六、英文提示词"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "nf-C5Y-9TR2d",
      "metadata": {
        "id": "nf-C5Y-9TR2d"
      },
      "source": [
        "**一、构建简单链**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "id": "urymAcXkTOQ2",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "urymAcXkTOQ2",
        "outputId": "97d1aee3-86a0-44cb-c534-6db4ffb6a11b"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Why did the bear bring a flashlight to the party?\\n\\nBecause he wanted to be the \"light\" of the bearbecue!'"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "prompt = ChatPromptTemplate.from_template(\n",
        "    \"tell me a short joke about {topic}\"\n",
        ")\n",
        "model = ChatOpenAI()\n",
        "output_parser = StrOutputParser()\n",
        "\n",
        "chain = prompt | model | output_parser\n",
        "\n",
        "chain.invoke({\"topic\": \"bears\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e34b6bd6",
      "metadata": {},
      "source": [
        "**2.1 构建简单文档数据库**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "id": "J-IpScccTHtL",
      "metadata": {
        "id": "J-IpScccTHtL"
      },
      "outputs": [],
      "source": [
        "vectorstore = DocArrayInMemorySearch.from_texts(\n",
        "    [\"harrison worked at kensho\", \"bears like to eat honey\"],\n",
        "    embedding=OpenAIEmbeddings()\n",
        ")\n",
        "retriever = vectorstore.as_retriever()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "id": "4nWE8nLjTJuT",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4nWE8nLjTJuT",
        "outputId": "65742853-5146-4a1c-fb0b-d1e456532b95"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[Document(page_content='harrison worked at kensho'),\n",
              " Document(page_content='bears like to eat honey')]"
            ]
          },
          "execution_count": 28,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "retriever.get_relevant_documents(\"where did harrison work?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "id": "hygSW50bTXzV",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hygSW50bTXzV",
        "outputId": "2f88f2d7-781d-41d0-c50e-fa18e0420c3d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[Document(page_content='bears like to eat honey'),\n",
              " Document(page_content='harrison worked at kensho')]"
            ]
          },
          "execution_count": 29,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "retriever.get_relevant_documents(\"what do bears like to eat\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "id": "-6iG51i5TZGJ",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "-6iG51i5TZGJ",
        "outputId": "29de378b-2151-43fc-d2fc-f6a1b8d16896"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Harrison worked at Kensho.'"
            ]
          },
          "execution_count": 31,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "template = \"\"\"Answer the question based only on the following context:\n",
        "{context}\n",
        "\n",
        "Question: {question}\n",
        "\"\"\"\n",
        "prompt = ChatPromptTemplate.from_template(template)\n",
        "\n",
        "chain = RunnableMap({\n",
        "    \"context\": lambda x: retriever.get_relevant_documents(x[\"question\"]),\n",
        "    \"question\": lambda x: x[\"question\"]\n",
        "}) | prompt | model | output_parser\n",
        "\n",
        "chain.invoke({\"question\": \"where did harrison work?\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aabda125",
      "metadata": {},
      "source": [
        "**3.2 使用RunnableMap**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "id": "JoLBmwfETg1L",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JoLBmwfETg1L",
        "outputId": "73127639-c19e-400c-e26e-8f68dfd93635"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'context': [Document(page_content='harrison worked at kensho'),\n",
              "  Document(page_content='bears like to eat honey')],\n",
              " 'question': 'where did harrison work?'}"
            ]
          },
          "execution_count": 33,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "inputs = RunnableMap({\n",
        "    \"context\": lambda x: retriever.get_relevant_documents(x[\"question\"]),\n",
        "    \"question\": lambda x: x[\"question\"]\n",
        "})\n",
        "\n",
        "inputs.invoke({\"question\": \"where did harrison work?\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "HqjgfGT8Tj2Z",
      "metadata": {
        "id": "HqjgfGT8Tj2Z"
      },
      "source": [
        "**3.1 单函数绑定**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "id": "_XZPKC1_Th6S",
      "metadata": {
        "id": "_XZPKC1_Th6S"
      },
      "outputs": [],
      "source": [
        "functions = [\n",
        "    {\n",
        "      \"name\": \"weather_search\",\n",
        "      \"description\": \"Search for weather given an airport code\",\n",
        "      \"parameters\": {\n",
        "        \"type\": \"object\",\n",
        "        \"properties\": {\n",
        "          \"airport_code\": {\n",
        "            \"type\": \"string\",\n",
        "            \"description\": \"The airport code to get the weather for\"\n",
        "          },\n",
        "        },\n",
        "        \"required\": [\"airport_code\"]\n",
        "      }\n",
        "    }\n",
        "  ]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "id": "0UPE0nWDTm3T",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0UPE0nWDTm3T",
        "outputId": "a4762623-c42f-4950-a21e-da73f7efe0c9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "AIMessage(content='', additional_kwargs={'function_call': {'name': 'weather_search', 'arguments': '{\\n  \"airport_code\": \"SFO\"\\n}'}})"
            ]
          },
          "execution_count": 35,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"human\", \"{input}\")\n",
        "    ]\n",
        ")\n",
        "model = ChatOpenAI(temperature=0).bind(functions=functions)\n",
        "\n",
        "runnable = prompt | model\n",
        "\n",
        "runnable.invoke({\"input\": \"what is the weather in sf\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bc6a3fd8",
      "metadata": {},
      "source": [
        "**3.2 多个函数绑定**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "id": "U2R9nKncTpy6",
      "metadata": {
        "id": "U2R9nKncTpy6"
      },
      "outputs": [],
      "source": [
        "functions = [\n",
        "    {\n",
        "      \"name\": \"weather_search\",\n",
        "      \"description\": \"Search for weather given an airport code\",\n",
        "      \"parameters\": {\n",
        "        \"type\": \"object\",\n",
        "        \"properties\": {\n",
        "          \"airport_code\": {\n",
        "            \"type\": \"string\",\n",
        "            \"description\": \"The airport code to get the weather for\"\n",
        "          },\n",
        "        },\n",
        "        \"required\": [\"airport_code\"]\n",
        "      }\n",
        "    },\n",
        "        {\n",
        "      \"name\": \"sports_search\",\n",
        "      \"description\": \"Search for news of recent sport events\",\n",
        "      \"parameters\": {\n",
        "        \"type\": \"object\",\n",
        "        \"properties\": {\n",
        "          \"team_name\": {\n",
        "            \"type\": \"string\",\n",
        "            \"description\": \"The sports team to search for\"\n",
        "          },\n",
        "        },\n",
        "        \"required\": [\"team_name\"]\n",
        "      }\n",
        "    }\n",
        "  ]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "id": "i9yES-wwTrYO",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i9yES-wwTrYO",
        "outputId": "c4e1ae94-d087-454b-a8c1-7fab9df2b790"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "AIMessage(content='', additional_kwargs={'function_call': {'name': 'sports_search', 'arguments': '{\\n  \"team_name\": \"patriots\"\\n}'}})"
            ]
          },
          "execution_count": 37,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model = model.bind(functions=functions)\n",
        "\n",
        "runnable = prompt | model\n",
        "\n",
        "runnable.invoke({\"input\": \"how did the patriots do yesterday?\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "300a9405",
      "metadata": {},
      "source": [
        "**4.1 使用早期模型格式化输出**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "id": "4lECkukwT4ua",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 107
        },
        "id": "4lECkukwT4ua",
        "outputId": "88c67d19-6d26-406a-9d26-08827794dc4e"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'\\n\\n{\\n    \"title\": \"Autumn Leaves\",\\n    \"author\": \"Emily Dickinson\",\\n    \"first_line\": \"The leaves are falling, one by one\"\\n}\\n\\n{\\n    \"title\": \"The Ocean\\'s Song\",\\n    \"author\": \"Pablo Neruda\",\\n    \"first_line\": \"I hear the ocean\\'s song, a symphony of waves\"\\n}\\n\\n{\\n    \"title\": \"A Winter\\'s Night\",\\n    \"author\": \"Robert Frost\",\\n    \"first_line\": \"The snow falls softly, covering the ground\"\\n}'"
            ]
          },
          "execution_count": 38,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "simple_model = OpenAI(\n",
        "    temperature=0,\n",
        "    max_tokens=1000,\n",
        "    model=\"gpt-3.5-turbo-instruct\"\n",
        ")\n",
        "simple_chain = simple_model | json.loads\n",
        "\n",
        "challenge = \"write three poems in a json blob, where each poem is a json blob of a title, author, and first line\"\n",
        "\n",
        "simple_model.invoke(challenge)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "35c60800",
      "metadata": {},
      "source": [
        "**早期模型不支持，会出现解码错误**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "id": "aX0oo-25T9hE",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 330
        },
        "id": "aX0oo-25T9hE",
        "outputId": "8425f5f7-5891-4448-d67d-be89208e05a9"
      },
      "outputs": [
        {
          "ename": "JSONDecodeError",
          "evalue": "Extra data: line 9 column 1 (char 125)",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mJSONDecodeError\u001b[0m                           Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-39-7b2363c45b31>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msimple_chain\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minvoke\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchallenge\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/base.py\u001b[0m in \u001b[0;36minvoke\u001b[0;34m(self, input, config)\u001b[0m\n\u001b[1;32m   2051\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2052\u001b[0m             \u001b[0;32mfor\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstep\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msteps\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2053\u001b[0;31m                 input = step.invoke(\n\u001b[0m\u001b[1;32m   2054\u001b[0m                     \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2055\u001b[0m                     \u001b[0;31m# mark each step as a child run\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/base.py\u001b[0m in \u001b[0;36minvoke\u001b[0;34m(self, input, config, **kwargs)\u001b[0m\n\u001b[1;32m   3505\u001b[0m         \u001b[0;34m\"\"\"Invoke this runnable synchronously.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3506\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"func\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3507\u001b[0;31m             return self._call_with_config(\n\u001b[0m\u001b[1;32m   3508\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_invoke\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3509\u001b[0m                 \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/base.py\u001b[0m in \u001b[0;36m_call_with_config\u001b[0;34m(self, func, input, config, run_type, **kwargs)\u001b[0m\n\u001b[1;32m   1244\u001b[0m             output = cast(\n\u001b[1;32m   1245\u001b[0m                 \u001b[0mOutput\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1246\u001b[0;31m                 context.run(\n\u001b[0m\u001b[1;32m   1247\u001b[0m                     \u001b[0mcall_func_with_variable_args\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1248\u001b[0m                     \u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m  \u001b[0;31m# type: ignore[arg-type]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/config.py\u001b[0m in \u001b[0;36mcall_func_with_variable_args\u001b[0;34m(func, input, config, run_manager, **kwargs)\u001b[0m\n\u001b[1;32m    324\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mrun_manager\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0maccepts_run_manager\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    325\u001b[0m         \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"run_manager\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrun_manager\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 326\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# type: ignore[call-arg]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    327\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    328\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/base.py\u001b[0m in \u001b[0;36m_invoke\u001b[0;34m(self, input, run_manager, config, **kwargs)\u001b[0m\n\u001b[1;32m   3381\u001b[0m                         \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mchunk\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3382\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 3383\u001b[0;31m             output = call_func_with_variable_args(\n\u001b[0m\u001b[1;32m   3384\u001b[0m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconfig\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrun_manager\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3385\u001b[0m             )\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/langchain_core/runnables/config.py\u001b[0m in \u001b[0;36mcall_func_with_variable_args\u001b[0;34m(func, input, config, run_manager, **kwargs)\u001b[0m\n\u001b[1;32m    324\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mrun_manager\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0maccepts_run_manager\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfunc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    325\u001b[0m         \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"run_manager\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mrun_manager\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 326\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# type: ignore[call-arg]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    327\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    328\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/lib/python3.10/json/__init__.py\u001b[0m in \u001b[0;36mloads\u001b[0;34m(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)\u001b[0m\n\u001b[1;32m    344\u001b[0m             \u001b[0mparse_int\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mparse_float\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    345\u001b[0m             parse_constant is None and object_pairs_hook is None and not kw):\n\u001b[0;32m--> 346\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0m_default_decoder\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    347\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mcls\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    348\u001b[0m         \u001b[0mcls\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mJSONDecoder\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/lib/python3.10/json/decoder.py\u001b[0m in \u001b[0;36mdecode\u001b[0;34m(self, s, _w)\u001b[0m\n\u001b[1;32m    338\u001b[0m         \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_w\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    339\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 340\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mJSONDecodeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Extra data\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    341\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mobj\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    342\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mJSONDecodeError\u001b[0m: Extra data: line 9 column 1 (char 125)"
          ]
        }
      ],
      "source": [
        "simple_chain.invoke(challenge)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "935a60b7",
      "metadata": {},
      "source": [
        "**4.2 较新的模型能够格式化输出**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "id": "oLFppvMMT-w7",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oLFppvMMT-w7",
        "outputId": "3fa0a0c1-3e9b-44ca-acaa-4b5fe5915c43"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'poem1': {'title': 'Whispers of the Wind',\n",
              "  'author': 'Emily Rivers',\n",
              "  'first_line': 'Softly it comes, the whisper of the wind'},\n",
              " 'poem2': {'title': 'Silent Serenade',\n",
              "  'author': 'Jacob Moore',\n",
              "  'first_line': 'In the stillness of night, a silent serenade'},\n",
              " 'poem3': {'title': 'Dancing Shadows',\n",
              "  'author': 'Sophia Anderson',\n",
              "  'first_line': 'Shadows dance upon the walls, a secret ballet'}}"
            ]
          },
          "execution_count": 40,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model = ChatOpenAI(temperature=0)\n",
        "chain = model | StrOutputParser() | json.loads\n",
        "\n",
        "chain.invoke(challenge)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c76fe472",
      "metadata": {},
      "source": [
        "**4.3 fallback机制**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "id": "mWQ6qEwSUA0y",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mWQ6qEwSUA0y",
        "outputId": "1ca395fa-976f-4f5d-c0f4-f2bcbd0ba325"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'poem1': {'title': 'Whispers of the Wind',\n",
              "  'author': 'Emily Rivers',\n",
              "  'first_line': 'Softly it comes, the whisper of the wind'},\n",
              " 'poem2': {'title': 'Silent Serenade',\n",
              "  'author': 'Jacob Moore',\n",
              "  'first_line': 'In the stillness of night, a silent serenade'},\n",
              " 'poem3': {'title': 'Dancing Shadows',\n",
              "  'author': 'Sophia Anderson',\n",
              "  'first_line': 'Shadows dance upon the moonlit floor'}}"
            ]
          },
          "execution_count": 41,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "final_chain = simple_chain.with_fallbacks([chain])\n",
        "\n",
        "final_chain.invoke(challenge)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7bd87734",
      "metadata": {},
      "source": [
        "**五、接口**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "id": "oenACTHsUFlm",
      "metadata": {
        "id": "oenACTHsUFlm"
      },
      "outputs": [],
      "source": [
        "prompt = ChatPromptTemplate.from_template(\n",
        "    \"Tell me a short joke about {topic}\"\n",
        ")\n",
        "model = ChatOpenAI()\n",
        "output_parser = StrOutputParser()\n",
        "\n",
        "chain = prompt | model | output_parser"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a5feb063",
      "metadata": {},
      "source": [
        "**5.1 invoke接口**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "id": "uhpMBbexUHK2",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "uhpMBbexUHK2",
        "outputId": "bd63b7f7-7d11-474e-bdd7-ad59a45323ff"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\""
            ]
          },
          "execution_count": 43,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chain.invoke({\"topic\": \"bears\"})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3ae6e2de",
      "metadata": {},
      "source": [
        "**5.2 batch接口**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "id": "ptSp8UhQUILV",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ptSp8UhQUILV",
        "outputId": "b3125d7e-c35b-4ea3-999f-75543f36ac34"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "WARNING:langchain_core.language_models.llms:Retrying langchain_community.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for gpt-3.5-turbo in organization org-GXqDV5Kltm74g4hGPYEjLdb0 on requests per min (RPM): Limit 3, Used 3, Requested 1. Please try again in 20s. Visit https://platform.openai.com/account/rate-limits to learn more. You can increase your rate limit by adding a payment method to your account at https://platform.openai.com/account/billing..\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "[\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\",\n",
              " 'Why did the frog take the bus to work?\\n\\nBecause his car got toad away!']"
            ]
          },
          "execution_count": 44,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chain.batch([{\"topic\": \"bears\"}, {\"topic\": \"frogs\"}])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "826b9ed7",
      "metadata": {},
      "source": [
        "**5.3 stream接口**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "id": "Ehj9MzGPUI_I",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ehj9MzGPUI_I",
        "outputId": "239b158c-6283-4892-d6f5-bcbe69329cee"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "Why\n",
            " don\n",
            "'t\n",
            " bears\n",
            " wear\n",
            " shoes\n",
            "?\n",
            "\n",
            "\n",
            "Because\n",
            " they\n",
            " have\n",
            " bear\n",
            " feet\n",
            "!\n",
            "\n"
          ]
        }
      ],
      "source": [
        "for t in chain.stream({\"topic\": \"bears\"}):\n",
        "    print(t)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c2858691",
      "metadata": {},
      "source": [
        "**5.4 异步接口**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "id": "xucFn07uUJ7Z",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "xucFn07uUJ7Z",
        "outputId": "54545335-949b-463d-b8e2-41bb7c1076c1"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "\"Why don't bears wear shoes?\\n\\nBecause they have bear feet!\""
            ]
          },
          "execution_count": 47,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response = await chain.ainvoke({\"topic\": \"bears\"})\n",
        "response"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}
